Translation lookaside buffer (tlb) flush systems and methods for defeating tlb poisoning attacks

ABSTRACT

TLB poisoning attacks take advantage of security issues of translation lookaside buffer (TLB) management on SEV processors in Secure Encrypted Virtualization (SEV) virtual machines (VMs). In various embodiments, a hypervisor may poison TLB entries between two processes of a SEV VM to compromise the integrity and confidentiality of the SEV VM. Variants of TLB poisoning attacks and end-to-end attacks are shown to be successful on both Advanced Micro Devices (AMD) SEV and SEV-Encrypted State (SEV-ES). Countermeasures for thwarting TLB poisoning attacks include hardware-enforced TLB flush processes and re-exec schemes that, among other things, prevent attackers from manipulating TLB entries and causing a privileged victim process to execute malicious code in an attempt to bypass a password authentication.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This patent application is related to and claims priority benefit under35 USC § 119(e) to co-pending and commonly-owned U.S. Pat. App. No.63/248,418, filed on Sep. 24, 2021, entitled “Translation LookasideBuffer (TLB) Poisoning Attacks On Secure Encrypted Virtualization,” andlisting Huibo Wang, Kang Li, Mengyuan Li, Yinqian Zhang, and YueqiangCheng as inventors (Docket No. 28888-2532P), which patent document isincorporated by reference herein in its entirety and for all purposes.

COPYRIGHT NOTICE

A portion of the disclosure in this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patentdocument, as it appears in the Patent and Trademark Office patent fileor records, but otherwise reserves all copyright rights whatsoever.

BACKGROUND A. Technical Field

The present disclosure relates generally to computer security. Moreparticularly, the present disclosure relates to attacks on SEVprocessors by an untrusted hypervisor and appropriate countermeasures.

B. Background

Computers have become almost completely ubiquitous. From smart phones,tablet computers, smart appliances, laptop computers, cloud computing,embedded devices, Internet of Things (IoT) devices, and other devices,computing devices are extensively used and have extensive access tocritical information. As their use and prevalence increases, they createincreasing concerns regarding computer security.

Advanced Micro Devices' (AMD's) SEV Secure Encrypted Virtualization(SEV) is an emerging technology of AMD server processors that providestransparent memory encryption and key management for virtual machines(VMs). SEV supports full VM encryption and, thus, makes portingapplications straightforward. By not trusting the underlying hypervisor,SEV forms a foundation for confidential computing. Despite its attemptto provide security, SEV has known vulnerabilities that may be exploitedby malicious individuals or entities.

Accordingly, what is needed are the discovery of and effectivecountermeasures against such vulnerabilities to enhance security-relatedcomputer technologies, such as confidential cloud computing.

BRIEF DESCRIPTION OF THE DRAWINGS

References will be made to embodiments of the disclosure, examples ofwhich may be illustrated in the accompanying figures. These figures areintended to be illustrative, not limiting. Although the disclosure isgenerally described in the context of these embodiments, it should beunderstood that it is not intended to limit the scope of the disclosureto these particular embodiments. Items in the figures may not be toscale.

FIG. 1 depicts TLB misuse across virtual Central Processing Units(vCPUs), according to embodiments of the present disclosure.

FIG. 2A illustrates (VMCB) switching steps used to bypass thehardware-enforced TLB flushes according to embodiments of the presentdisclosure.

FIG. 2B depicts changing a vCPU's CPU affinity without TLB flush,according to embodiments of the present disclosure.

FIG. 3 depicts an overview of an exemplary attack steps to bypasspassword authentication, according to embodiments of the presentdisclosure.

FIG. 4A and FIG. 4B depict variations of the virtual address oftestcrypto for ten connections per second and one connection per second,respectively, according to embodiments of the present disclosure.

FIG. 5 is a flowchart illustrating a hypervisor-controlled attackprocess for manipulating a privileged victim process's execution tobypass password authentication, according to embodiments of the presentdisclosure

FIG. 6 is a flowchart illustrating a non-hypervisor-controlledtranslation lookaside buffer TLB poisoning process for attacking anetwork application, according to embodiments of the present disclosure.

FIG. 7 is a countermeasure against a TLB poisoning attack, according toembodiments of the present disclosure.

FIG. 8 is a hardware-enforced TLB flush process for mitigatinghypervisor-controlled TLB poisoning attacks, according to embodiments ofthe present disclosure

FIG. 9 depicts a simplified block diagram of a computingdevice/information handling system, according to embodiments of thepresent disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following description, for purposes of explanation, specificdetails are set forth in order to provide an understanding of thedisclosure. It will be apparent, however, to one skilled in the art thatthe disclosure can be practiced without these details. Furthermore, oneskilled in the art will recognize that embodiments of the presentdisclosure, described below, may be implemented in a variety of ways,such as a process, an apparatus, a system, a device, or a method on atangible computer-readable medium.

Components, or modules, shown in diagrams are illustrative of exemplaryembodiments of the disclosure and are meant to avoid obscuring thedisclosure. It shall be understood that throughout this discussion thatcomponents may be described as separate functional units, which maycomprise sub-units, but those skilled in the art will recognize thatvarious components, or portions thereof, may be divided into separatecomponents or may be integrated together, including, for example, beingin a single system or component. It should be noted that functions oroperations discussed herein may be implemented as components. Componentsmay be implemented in software, hardware, or a combination thereof.

Furthermore, connections between components or systems within thefigures are not intended to be limited to direct connections. Rather,data between these components may be modified, re-formatted, orotherwise changed by intermediary components. Also, additional or fewerconnections may be used. It shall also be noted that the terms“coupled,” “connected,” “communicatively coupled,” “interfacing,”“interface,” or any of their derivatives shall be understood to includedirect connections, indirect connections through one or moreintermediary devices, and wireless connections. It shall also be notedthat any communication, such as a signal, response, reply,acknowledgment, message, query, etc., may comprise one or more exchangesof information.

Reference in the specification to “one or more embodiments,” “preferredembodiment,” “an embodiment,” “embodiments,” or the like means that aparticular feature, structure, characteristic, or function described inconnection with the embodiment is included in at least one embodiment ofthe disclosure and may be in more than one embodiment. Also, theappearances of the above-noted phrases in various places in thespecification are not necessarily all referring to the same embodimentor embodiments.

The use of certain terms in various places in the specification is forillustration and should not be construed as limiting. A service,function, or resource is not limited to a single service, function, orresource; usage of these terms may refer to a grouping of relatedservices, functions, or resources, which may be distributed oraggregated. The terms “include,” “including,” “comprise,” “comprising,”or any of their variants shall be understood to be open terms, and anylists of items that follow are example items and not meant to be limitedto the listed items. A “layer” may comprise one or more operations. Theuse of memory, database, information base, data store, tables, hardware,cache, and the like may be used herein to refer to system component orcomponents into which information may be entered or otherwise recorded.A set may contain any number of elements, including the empty set.

Any headings used herein are for organizational purposes only and shallnot be used to limit the scope of the description or the claims. Eachreference/document mentioned in this patent document is incorporated byreference herein in its entirety.

It shall be noted that any experiments and results provided herein areprovided by way of illustration and were performed under specificconditions using a specific embodiment or embodiments; accordingly,neither these experiments nor their results shall be used to limit thescope of the disclosure of the current patent document.

It is noted that although embodiments described herein may be within thecontext of software-based side channel attacks, aspects of the presentdisclosure are not so limited. Accordingly, aspects of the presentdisclosure may be applied or adapted for use in hardware-based attacks,including side channel attacks, and other contexts.

In this document, the term “page walk” and “page table walk” are usedinterchangeably. Similarly, the terms “system physical address” and“host physical address;” “host” and “system;” “idle loop” and” “emptyloop;” “world switch,” “context switch,” and “internal switch;” “hostOS” and “system OS;” and “victim VM” and” “guest VM” may be usedinterchangeably. The term “guest” refers to any operating system that isvirtualized, and the term “hypervisor” refers to any platform layer thedecouples an operating system from its underlying hardware. A “nestedpage table” refers to a page table that comprises a translation from aguest physical address to a system physical address or host physicaladdress.

A. General Introduction

AMD's Secure Encrypted Virtualization (SEV) is a security feature of AMDVirtualization (AMD-V) that prevents privileged cloud providers frommanipulating or inspecting data and applications of cloud tenants. SEVsupports full VM encryption through a hardware memory encryption engineand a secure co-processor (e.g., AMD-SP) that transparently manageshardware encryption keys. Compared to Intel's Software Guard Extension(SGX), SEV is more advantageous in its ease of adoption, without theneed of altering software to be protected. So far, SEV has been adoptedin Azure Cloud and Google Cloud as a backend of confidential cloudcomputing services.

Nonetheless, numerous works have shown that SEV is vulnerable in severalaspects: First, the VMCB used to store register values and controlinformation is not encrypted during context switch, allowing a malicioushypervisor to manipulate or inspect register values of guest VMs, whichmay lead to a breach of confidentiality or integrity of guest VMs. Tocounter these threats, AMD released a second generation of SEV, SEVEncrypted State (SEV-ES) in which register states in VMCB are encryptedand saved in the VM Save Area (VMSA) during a world switch. Necessaryregister values are passed through a Guest-Hypervisor CommunicationBlock (GHCB), which is not encrypted.

Second, neither SEV nor SEV-ES protect the integrity of encrypted memoryand nested page tables (NPT). Therefore, a malicious hypervisor mayreplay encrypted memory blocks or change the mapping of guest memorypages in NPTs to breach the security of SEV and SEV processors. Tomitigate these attacks, AMD released a third generation of SEV, SEVSecure Nested Paging (SEV-SNP), which uses a Reverse Map Table (RMP) anda page validation mechanism to prevent malicious modification of NPTs bytracking memory ownership. According to AMD, SEV-SNP provides strongintegrity protection for guest VMs to mitigate such attacks.

Third, SEV, including SEV-ES and SEV-SNP, allows an untrusted hypervisorto manage the address space identifier (ASID), which is used to controlthe VM's accesses to the encrypted memory. AMD adopted a“security-by-crash” design, which assumes that mismatch in ASIDs couldlead to VM crashes and, hence, guarantees the security of the guest VMs.However, ASIDs have been exploited in Crossline attacks, which leveragethe short time window before a VM crashes to leak secret data throughpage faults or to execute instructions that form decryption andencryption oracles.

Outlined herein are embodiments of a new category of security attacksagainst SEV, namely TLB poisoning attacks that enable an adversary whocontrols the hypervisor to manipulate TLB entries that are sharedbetween two processes of the same SEV VM. One root cause of TLBpoisoning attacks is that, by design, the hypervisor is in control ofthe TLB flushes in SEV. Specifically, because TLB is tagged with ASIDsto distinguish TLB entries used by different entities, unnecessary TLBflushes may be avoided during the world switches (VMEXIT and VMRUNbetween the guest VM and the hypervisor) or context switches (contextswitches between the process hosting the guest VM's current vCPU andother processes). As it is difficult for the CPU hardware to determinewhether to flush the entire TLB or only TLB entries with certain ASIDs,the TLB flush is solely controlled by the hypervisor. The hypervisor mayinform the CPU hardware to fully (or partially) flush the TLB, bysetting the TLB control field in the VMCB, to take effect after VMRUN.As a result, an adversary may intentionally skip TLB flushes to cause avictim process of the victim SEV VM to use TLB entries injected byanother process of the same VM.

Two exemplary scenarios of TLB poisoning attacks are considered: (1)using an unprivileged attacker process running in the targeted SEV VM,an adversary may poison TLB entries used by a privileged process andalter its execution; and (2) without the help of a process that isdirectly controlled by the adversary, the adversary may still exploitthe misuse of TLB entries on a network-facing process (also not underthe adversary's control) that share the same (or similar) virtualaddress space with a targeted process and bypass authentication checks.To demonstrate the feasibility of these two attack scenarios, twoend-to-end attacks against two SSH servers were performed on an AMDEPYC™ Zen processor supporting SEV-ES.

Embodiments presented herein comprise the following contributions to thesecurity of AMD SEV processors and confidential computing technology ingeneral:

First, they demystify AMD SEV's TLB management mechanisms and identify asevere design flaw of TLB isolation that leads to misuse of TLBs underthe assumption of a malicious or compromised hypervisor or otheruntrusted privileged software.

Second, they present a novel category of attacks against SEV, namely TLBpoisoning attacks, which manipulate the TLB entries shared by twoprocesses within the same SEV VM and breach the integrity andconfidentiality of one of the processes.

Third, they demonstrate two end-to-end TLB poisoning attacks againstSEV-ES-protected VMs. One attack shows the feasibility of manipulatingTLB entries to change code execution of the victim process. Anotherattack provides an example of stealing secret data from the victimprocess by a process, which is not controlled by the adversary, by usingshared TLB entries.

B. General Background Information

Some background information about SEV's memory and TLB isolation ispresented next.

Secure Encrypted Virtualization (SEV): AMD's memory encryption featurefor AMD-V, SEV aims to produce a confidential VM environment in thepublic cloud and protect VMs from the privileged but untrustworthy cloudhosts (e.g., a hypervisor). SEV is built on an on-chip encryption systemcomprising an ARM Cortex-A5 co-processor and AES encryption engines. Theco-processor, also known as AMD-SP, stores and maintains a set of VMencryption keys (K_(vek)) which is uniquely assigned to each SEV-enabledVM. The K_(vek) in the co-processor cannot be accessed by either theprivileged hypervisor or the guest VM itself. The AES encryption engineautomatically encrypts all data in the memory, and decrypts them in theCPU by using the correct K_(vek).

Nested Page Tables. AMD adopts two-level of page tables to help thehypervisor manage the SEV VM's memory mapping. The upper-level pagetable, called the guest page table (gPT), is part of the guest VM'sencrypted memory. It is maintained by the guest VM and is usually a4-level page table that translates the guest virtual address (gVA) tothe guest physical address (gPA). Moreover, a Guest Page Fault (gPF)caused by the gPT walk is trapped and handled by the guest VM. Thelower-level page table, called NPT or host page table (hPT), translatesgPA to system physical address (sPA) and is maintained by thehypervisor. The NPT structure gives the SEV VM the ability to configurethe memory pages' encryption states. By changing the C-bit (Bit 47 inthe page table entry) to 1 or 0, the states of the guest VM's memorypage can either be private (encrypted with his K_(vek)) or shared(encrypted with the hypervisor's K_(vek)) The gPT and all instructionpages are forced to be private states no matter the value of C-bit.

Moreover, Nested Page Faults (NPFs) may be triggered by the hardwareduring the NPT walk. According to the NPF event, the hypervisor can grabuseful information that could reflect the behavior of a program and,therefore, leak sensitive information, including the gPA of the NPT andthe NPF error code. This forms a well-known controlled-channel attack,which compromises SEV's confidentiality and integrity.

Address Space Layout Randomization (ASLR). ASLR is a widely usedspectrum protection technique that randomizes virtual memory areas(VMAs) of a process to defend against memory corruption attacks. Thisdefense mechanism prevents attackers from directly learning a pointer'svirtual address and forces them to rely on software vulnerabilities orside-channel attacks to locate the randomized virtual address. Differentoperating systems have different ASLR implementations. For example, a64-bit Linux system usually exhibits 28-bits of ASLR entropy forexecutables, while Windows 10 exhibits only 17-19 bits of ASLR entropyfor executables.

Translation Lookaside Buffer and Address Space Identifier (ASID). TLB isa caching hardware inside the chip's memory-management unit (MMU). Aftera successful page table walk, the mapping from the virtual address tothe system address is cached in TLB. For a nested page table on SEV, themapping of the gVA and the sPA is cached in the TLB. During a page tablewalk, given a guest CR3 (gCR3) and a host CR3 (hCR3), the hardwareautomatically translates a gVA to an sPA using the two-level page tablesdespite the gPT and the NPT being encrypted by different K_(vek)s.AMD-SP uses ASID to uniquely identify the SEV-enabled VM and itsK_(vek). ASID is also part of the tag for both cache lines and TLBentries.

C. Understanding and Demystifying SEV's TLB Isolation Mechanisms

In this section, an understanding of TLB isolation mechanisms used inAMD-V for both non-SEV VMs and SEV-enabled VMs is briefly set forth.

1. TLB Management for Non-SEV VM Embodiments

To avoid frequent TLB flushes during VM world switches, AMD introducedASIDs in TLB entries. ASID 0 is reserved for the hypervisor, and theremainder of ASIDs are used by the VM. The range of the ASID pool can bedetermined by CPUID 0x8000000a[EBX]. TLB is tagged with the ASIDs ofeach VM and the hypervisor, which avoids flushing the entire TLB at theworld switch and also prevents misuse of the TLB entries belonging toother entities.

The TLB management algorithm for non-SEV VMs has been explored byexamine the source code of AMD SVM. Specifically, the hypervisor isresponsible for maintaining the uniqueness and the freshness of the ASIDin each logical core of the machine. For each logical core, thehypervisor stores the most recently used ASID in the svm_cpu_data datastructure. Before each VMRUN of a vCPU of a non-SEV VM, the hypervisorchecks whether the CPU affinity of the vCPU has changed by comparing theASID stored in its VMCB with the most recently used ASID of this logicalcore. If a mismatch is observed, which means either the vCPU was notrunning on this logical core before the current VM Exit (VMEXIT) or morethan one vCPU sharing the same logical core concurrently, the hypervisorassigns an incremental and unused ASID to this vCPU. In either of thesecases, incrementing the ASID ensures the residual TLB entries cannot bereused. Otherwise, no TLB flushing may be needed and the vCPU can keepits ASID and reuse its TLB entries after VMRUN.

The hypervisor is in charge of enforcing TLB flushes under certainconditions. For example, when the recently used ASID exceeds the maxASID range on the logical core, a complete TLB flush for all ASIDs maybe required. To flush TLBs, the hypervisor sets the TLB_CONTROL bits inTLB_CONTROL field (058h) of the VMCB during VMEXITs. With differentvalues of bits 39:32 of TLB_CONTROL, the hardware may perform differentoperations on the TLB:

-   -   TLB_CONTROL_DO_NOTHING (00h). The hardware does nothing.    -   TLB_CONTROL_FLUSH_ALL_ASID (01h). The hardware flushes the        entire TLB.    -   TLB_CONTROL_FLUSH_ASID (03h). The hardware flushes all TLB        entries whose ASID is equal to the ASID in the VMCB.    -   TLB_CONTROL_FLUSH_ASID_LOCAL (07h). The hardware flushes this        guest VM's non-global TLB entries.    -   Other values. All other values are reserved. As a result, these        values may cause problems when resuming a guest VM process.

After each VMRUN, hardware checks these bits and performs thecorresponding actions. The hypervisor is in charge of informing thehardware to flush TLBs and maintaining TLB isolation. Hardware may alsoautomatically perform a partial TLB flush without triggering a specialVMEXIT, e.g., when observing context switches or MOV-to-CR3instructions. In such cases, only the TLB entries tagged with thecurrent ASID (either in the guest ASID or the hypervisor ASID) areflushed.

2. Demystifying SEV's TLB Management Embodiments

The TLB management for SEV VMs and non-SEV VMs is slightly different.The ASIDs of SEV VMs remain the same during their lifetime. Therefore,instead of dynamically assigning an ASID to a vCPU, all vCPUs of thesame SEV VM have the same ASID. At runtime, TLB flush is stillcontrolled by the hypervisor. Especially, KVM records the last residentCPU core of each vCPU. For each CPU logical core, KVM also records theVMCB of the last running vCPU (sev_vmcbs [asid]) for each ASID. Beforethe hypervisor resumes a vCPU via VMRUN, it sets the TLB control fieldin the VMCB to the value of TLB_CONTROL_FLUSH_ASID when (1) this vCPUwas not run on this core before, or (2) the last VMCB running on thiscore with the same ASID is not the current VMCB. This enforces theisolation between two vCPUs of the same SEV VM. The code is listed inListing 1. However, if the hypervisor chooses not to set the TLB controlfield, no TLB entries will be flushed.

Listing 1: Code snippet of pre_sev_run() embodiment 1 structsvm_cpu_data *sd = per_cpu (svm_data, cpu); 2 int asid =sev_get_asid(svm -> vcpu.kvm ); 3 pre_sev_es_run(svm); 4 svm ->vmcb ->control.asid = asid; 5 // No CPU affinity change and No VMCB change 6 if(sd ->sev_vmcbs[asid] == svm->vmcb && 7  svm->vcpu.arch.last_vmentry_cpu == cpu) 8 return ; 9 // Otherwise, flushthe TLB tagged with the ASID 10 sd -> sev_vmcbs[asid] = svm -> vmcb ; 11svm ->vmcb ->control.tlb_ctl = TLB_CONTROL_FLUSH_ASID; 12vmcb_mark_dirty(svm ->vmcb, VMCB_ASID); 13 }

Experiments to demystify TLB tags. According to the AMD manual, an ASIDis part of a TLB tag. But it is unclear what the remaining parts of thetag are. Experiments were conducted to explore the structure of TLBtags. Specifically, it was examined whether vCPUs' TLB entries onco-resident logical cores influence each other, and whether TLB entriesfrom different VM modes (non-SEV, SEV, or SEV-ES) influence each other.Experimental settings were similar. To explore TLB isolation betweenco-resident logical cores, the ASID of two vCPUs were manually set totwo co-resident logical cores of the same physical core. To explore TLBisolation between VMs with different VM modes (e.g., SEV and non-SEV), anon-SEV VM and a SEV/SEV-ES VM were configured on the same logical core,and the non-SEV VM's ASID was set to be identical to the SEV/SEV-ES VM'sASID. In both cases, the TLB flush was skipped to check whether TLBpoisoning could be observed (using steps in Section D.2.1). In neitherof the two cases was TLB poisoning observed. Therefore, one mayconclude:

-   -   ASID is a part of the TLB tag, which provides TLB isolation for        TLB entries with different ASIDs.    -   Logical Core ID is a part of the TLB tag, which provides TLB        isolation for TLB entries on the same physical core but        different logical cores.    -   VM mode is a part of the TLB tag. Even if a non-SEV VM has the        same ASID as a SEV or SEV-ES VM, the TLB tag field contains        information about the VM's mode, which isolates TLB entries from        VMs in different modes.

Besides these components, it may further be assumed that the C-bit inthe guest page table (gC-bit) and the C-bit in the nested page table(nC-bit) are also part of the TLB tag. The reason is that when addresstranslation bypasses the page table walk, the values of the gC-bit andnC-bit are still required for the processor to determine which ASID topresent to AMD-SP if memory encryption is needed. However, there is nodirect evidence to ascertain the exact C-bit tag format in TLB entries.There may be no way to empirically affirm whether, for instance, onlyone C-bit or both C-bits are in the TLB tag.

3. TLB Flush Rules for SEV VMs Embodiments

The TLB flush rules for SEV/SEV-ES VMs in both hardware-enforced TLBflush and the hypervisor-coordinated TLB flush are summarized inTable 1. The hardware-enforced TLB flush rules cannot be skipped, whilethe hypervisor-coordinated TLB flush may be skipped by a malicioushypervisor, which is the root cause of the TLB poisoning attack.

TABLE 1 TLB flush rules. The World column indicates whether the eventhappens in host world or the guest world; TLB Tag represents the TLBentry’s ASID to be flushed-the host’s ASID is 0 and the SEV VM’s ASID isN; Forced indicates whether the TLB flush is forced by the hardware orcontrollable by the hypervisor. * highlights a special case, in whichwhen the world switch happens between two vCPUs, the TLB tagged with 0is forced to be flushed while the TLB tagged with N is flushed under thecontrol of the hypervisor. World Events TLB Tag Forced Host/GuestMOV-to-CR3, Context-switch 0/N ✓ Host/Guest Update Cr0.PG 0/N ✓Host/Guest Update CR4 (PGE, PAEm and PSE) 0/N ✓ Host Address translationRegisters All ✓ Host Activate an ASID for SEV VM N ✓ Host Deactivate anASID for SEV VM N X Host ASID exceeds ASID pool range All X Host TwovCPUs switch 0 + N* ✓ + X* Host Change vCPU’s CPU affinity N X

Hardware-enforced TLB flushes. All TLB entries are flushed when there isSystem Management Interrupt (SMI), Returning from System Management(RSM), Memory-Type Range Register (MTRR), and I/O Range Registers (IORR)modifications or MSRs access related to address translation,irrespective of ASID. At the same time, hardware will automaticallyflush TLB tags with the current ASID when observing activities likeMOV-to-CR3, context switches, updates of CR0.PG, CR4.PGE, CR4.PAEm, andCR4.PSE. Hardware will also force a TLB flush when the hypervisor wantsto activate an ASID for a SEV VM.

Hypervisor-coordinated TLB flushes. There are mainly two cases ofhypervisor-coordinated flushes in TLB management: (1) when differentVMCBs with the same ASID (different vCPUs of the same SEV VM) are to runon the same logical core; and (2) when the VMCB to be run was executedon a different logical core prior to the current VMRUN.

D. Attack Primitives Embodiments

In this section, the threat models considered in this patent documentare discussed, and three attack primitives are introduced: TLB misuseacross vCPUs (Section D.2), TLB misuse within the same vCPU (SectionD.3), and a covert data transmission channel between the hypervisor anda process in the victim VM that is under the adversary's control(Section D.4).

1. Threat Model Embodiments

It is assumed that a platform is hosted by a hypervisor that iscontrolled by an adversary, the victim VM is an SEV-ES-enabled VM and,thus, protected by all SEV-ES features. It is further assumed that ASLRis enabled in the victim VM. An unprivileged attacker process controlledby the adversary that is running in the victim VM does not have accessto the kernel or learn sensitive information from prods. The attackerprocess needs no capabilities to perform network communication. It isnoted that the assumption of having an attacker process running in thevictim VM can be weakened (see, Section F). The victim process may beany process in the victim VM other than the attacker processes. It isalso assumed that the adversary can learn the virtual address range ofthe victim VM via any other attack, e.g., a CrossLine attack.

2. TLB Misuse Across vCPUs Embodiments

When the victim VM has more than one vCPU, the attacker process and thevictim process can run on different vCPUs. The vCPU running the attackerprocess is herein referred to as the attacker vCPU, and the vCPU runningthe victim process is referred to as the victim vCPU. In one or moreembodiments, an adversary may misuse TLB entries by skipping the TLBflush during the context switch of these two vCPUs. Two examplesillustrate how this may be exploited to breach the integrity andconfidentiality of a victim process:

a) TLB Poisoning Embodiments

As illustrated in FIG. 1 , an attacker process may poison TLB entries toalter the execution of the victim process, e.g., by using the followingsteps:

Step-I: In one or more embodiments, the victim process is suspendedbefore executing an instruction at address VA₀. This may be achieved,e.g., by manipulating page table entries (PTEs) to trigger NPFs. It isnoted that the content of this instruction is not relevant to thisattack.

Step-II: The hypervisor schedules the attacker vCPU to the same logicalcore as the victim vCPU, and the TLB control field is set toTLB_CONTROL_FLUSH_ASID (03h) to flush the TLB entries with the SEV VM'sASID.

Step-III: The hypervisor then instructs the attacker process to run aninstruction sequence “mov $0x2021, % rax; CPUID” also at address VA₀.The CPUID instruction will trigger a VMEXIT. During the VMEXIT, theattacker vCPU is paused, and the victim vCPU is scheduled to run withoutflushing the TLB entries.

Step-IV: When the victim process executes the instruction at VA₀, aVMEXIT due to CPUID can be observed with the % rax value set to 0x2021in the GHCB. This means the victim process has been successfully trickedto execute the same instruction as the attacker process at VA₀, becauseit reuses the TLB entry poisoned by the attacker process.

b) Secret Leaking Embodiments

The second example illustrates that the attacker process may read thevictim process's memory space directly.

Step-I: The attacker process may use mmap( ) syscall to pre-map a datapage such that the virtual address VA₀ points to a data region on thatpage.

Step-II: The victim process may be scheduled to run and access thememory at address VA₀, which may be either an instruction fetch or adata load. This loads a TLB entry into the TLB.

Step-III: The hypervisor may de-schedule the victim vCPU, and theattacker vCPU may be scheduled to run on the same logical core. Thehypervisor may set the TLB control field of the attacker's VMCB toTLB_CONTROL_DO_NOTHING (00h) such that no TLB entries are flushed.

Step-IV: After being scheduled to run and loading data from VA₀, it isobserved that the attacker process successfully loads the data from thevictim's address space, thereby, compromising the victim'sconfidentiality. This is because the TLB entries created by the victimprocess are reused by the attacker process.

3. TLB Misuse within the Same vCPU Embodiments

When the victim VM has only one vCPU, the attacker process shares thevCPU with the victim process. In this case, TLB misuse is lessstraightforward. The TLB flush rules illustrated in Section C.3 (above)suggest that the hardware will automatically flush the entire TLB taggedby the victim VM vCPU's ASID when there is an internal context switch inthe guest VM, which leaves no chance for the hypervisor to skip the TLBflush. As such, the hypervisor cannot directly misuse the TLB entriesbetween two processes within the same vCPU. To address this challenge,presented herein are novel VMCB-switching embodiments to bypass thehardware-enforced TLB flush during the internal context switch.

a) Bypassing Hardware-Enforced TLB Flushes Embodiments

One key to bypassing the hardware-enforced TLB flush is to reserve theattacker process's TLB entries on one CPU core and then to migrate thevCPU to another CPU core. The internal context switch between the victimprocess and the attacker process is then performed on the second CPUcore, which automatically flushes all TLB entries on the second logicalcore. Because the hypervisor isolates the first CPU core to preventother processes from evicting its TLB entries, the TLB entries of theattacker processes are hence preserved. The hypervisor then migrates thevCPU back, with the victim process executing on it. The victim processwill then misuse the TLB entries poisoned by the attacker process.

The challenges for bypassing the hardware-enforced TLB flush aretwo-fold: First, changing the vCPU affinity inside the victim VM leadsto TLB flush for both the victim and attacker processes; however, suchTLB flush can only be performed by a privileged process. Second,changing the CPU affinity outside the victim VM, i.e., from thehypervisor side, may easily evict the reserved TLB entries. Thus,traditional CPU schedule methods, such as taskset or sched_setaffinity,may not work.

b) VMCB-Switching Embodiments

FIG. 2A and FIG. 2B illustrate how VMCB-switching may be used to bypassthe hardware-enforced TLB flushes, according to embodiments of thepresent disclosure. FIG. 2A illustrates VMCB-switching steps, and FIG.2B illustrates depicts changing a vCPU's CPU affinity without TLB flush.

Step-I: In one or more embodiments, the hypervisor first isolates thetarget vCPU hosted in a hypervisor process HP₁ on logical core LC₁ andprevents other processes from accessing LC₁, as well as its co-residentlogical core on the same physical core. The hypervisor may also reserveanother logical core LC₂ with an idle hypervisor process HP₂, e.g., toensure that other processes will not evict the reserved TLB entries.

Step-II: After an attacker process poisons the targeted TLB entries, thehypervisor may trap the vCPU into an empty loop during an VMEXIT.Meanwhile, the hypervisor may let the idle process HP₂ on LC₂ to resumethe attacker vCPU using its VMCB, VMSA pointer, and NPT structures. Thisis possible because the states of the attacker vCPU (e.g., registers,ASID, Nested CR3) are stored in the DRAM in encrypted form using eitherthe hypervisor's memory encryption key (e.g., VMCB, NPT) or the guestVM's VM encryption key (e.g., VMSA). After resuming the attacker vCPU onLC₂, there are no valid TLB entries on LC₂, but the attacker processinside the attacker vCPU can continue execution after page table walks.

Step-III: The hypervisor may trap and trace changes to gCR3 to monitorthe internal context switches on the attacker vCPU. Specifically, thehypervisor may intercept TRAP_CR3_WRITE_VMEXIT and extract the gCR3value in the EXITINFO1 field of VMCB. Since the inner context switchoccurs on LC₂, no hardware-enforced TLB flush is triggered on LC₁ and,thus, the attacker process's TLB entries are preserved on LC₁.

Step-IV: After observing that a context switch from the attacker processto the victim process is scheduled, the hypervisor may switch theattacker vCPU back to LC₁, e.g., following a similar method as describedin Step-II. The hypervisor may stop HP₂ on LC₂ and release HP₁ on LC₁from the empty loop.

Step-V: After resuming execution on LC₁, the victim process first triesto execute its next instruction pointed by RIP in VMSA via a TLB lookup.The preserved TLB entries on LC₁ are unconditionally trusted by thehardware. After the victim process uses the attacker's TLB entries toexecute instructions, some remaining TLB entries belonging to theattacker process may potentially disturb the execution of the victimprocess afterwards. Thus, the hypervisor can choose to perform a totalTLB flush.

It is noted that the attacker process and the hypervisor may also breachconfidentiality of the victim process in a reversed way, e.g., where thehypervisor reserves the victim process's TLB entries and lets theattacker process reuse TLB entries to exfiltrate secrets from thevictim's address space.

4. CPUID-Based Covert Channel Embodiments

A third primitive embodiment was built for transmitting data between ahypervisor and an attacker process in the victim VM that is under theadversary's control. To do so, a CPUID-based covert channel was built sothat network communication was not required. The adversary-controlledprocess may execute CPUID instructions to receive data or pass data tothe hypervisor. In one or more embodiments, to send data to thehypervisor, the attacker process may trigger a CPUID with a reserved RAXvalue (e.g., 1234) to initiate a data transfer. The attacker process maythen repeatedly trigger CPUID with RAX registers populated with the datato be transferred. Similarly, to receive data from the hypervisor, theattacker process may trigger a CPUID with another reserved RAX value(e.g., 1235). The hypervisor may retrieve the value of RAX and pass thedata to GHCB's RAX field before VMRUN. The attacker process may thenread the value of RAX after the CPUID instruction. Data received fromthe covert channel may be used as commands; the attacker processperforms pre-defined actions (e.g., mmap memory page and read certainvirtual address) in accordance with the command received. Inexperiments, the maximum transmission speed was 1.854 MB/s when usingthe 8-byte RAX register for data transmission. Other covert channelsthat make use of cache timing or AMD's way predictor may also be adoptedas covert channels but may be less robust.

E. TLB Poisoning with Assisting Processes Embodiments

This section discusses a first variant of TLB poisoning attackembodiments, which are assisted by an unprivileged attacker processrunning in the victim VM. Following the threat model described inSection D.1 above, it is assumed that the attacker process isunprivileged and has limited access to system resources, such as procfs,networking, or any privileged system capabilities. This is practical,e.g., in scenarios where the adversary has an unprivileged user accounton the victim VM, or an application with security vulnerabilities isremotely exploitable by the adversary. To simplify the attack, it isassumed that ASLR is disabled on the victim VM or that the attackerprocess can learn the VMA of the victim process. In practice, theattacker process may break ASLR using a CROSSLINE attack or any otherexisting method.

1. OpenSSH Embodiments

It is shown that, with the help of an unprivileged attacker processwithin a guest VM, an adversary may poison the TLB entries of aprivileged victim process and then control its execution. The attack maybe applied to OpenSSH and be used to bypass password authentication.

a) OpenSSH's Process Management Embodiments

Once a secure shell daemon (sshd) process, P_(d), is launched during asystem boot phase, P_(d) runs in the background and listens toconnections on SSH ports (e.g., port 22). The address space of P_(d) isdefined in the kernel by VMA data structures. Upon receiving aconnection, P_(d) forks an sshd child process, P_(c), which performs aprivilege separation (or privsep) by spawning another unprivilegedprocess, P_(n), to deal with the network transmission, but P_(d) keepsthe root privilege to itself to act as a monitoring process. Once a useris successfully authenticated, P_(n) is terminated, and a new processP_(u) is created under the new user's username. As discussed furtherbelow, in one or more TLB poisoning attack embodiments, an attackerprocess may take advantage of a victim process, which is the privilegedchild sshd process, P_(c), to poison TLB entries.

b) Password Authentication Bypass Embodiments

The adversary may first initialize a SSH connection to a target VM andmonitor gCR3 changes, e.g., by setting the CR3_WRITE_TRAP intercept bitin its VMCB. Once the SEV-ES VM receives the SSH packet from theadversary, the adversary may immediately observe a context switch (e.g.,gCR3 change). The new process to run is the sshd child process P_(c). Inthis way, the adversary may identify the gCR3 of P_(c).

(1) Locate the shared library. The attacker process may first help theadversary to locate the gPA of the shared library. In one or moreembodiments, pam_authenticate( ) which is a function of the sharedlibrary libpam.so.0 and is used by sshd for password authentication, maybe targeted. pam_authenticate( ) returns 0 if the authenticationsucceeds. An adversary may use the attacker process to help locate thegPA of pam_authenticate( ) (denoted gPA_(pam)), e.g., by firstsynchronizing with the two colluding entities using the covert channel(described in Section D.4) and then causing pam_authenticate( ) to becalled by the attacker process. The hypervisor may learn gPA_(pam) bytriggering NPFs.

(2) Track the victim's execution. The adversary may clear the Presentbit of all pages and monitor NPFs after intercepting his SSH packet withthe incorrect password. If an NPF of gPA_(pam) is observed, thisindicates to the adversary that the victim process is going toauthenticate the password by calling pam_authenticate( ). The adversarymay then pause the victim process by trapping the victim in thegPA_(pam) NPF handler. This may be used to provide a time window for theattacker process to poison the TLB entries. It is noted that this stepmay be useful in real attacks as it provides an opportunity for theattacker process to poison the TLB entries right before the victimprocess accesses those poisoned TLB entries, i.e., before the poisonedTLB entries are evicted by other activities.

(3) Poison TLB entries. The adversary may then poison the TLB entries ofthe victim. Assuming that the virtual address of the instruction pagecontaining pam_authenticate( ) in P_(c) is gVA_(pam) and that theadversary learns gVA_(pam) in advance, gVA_(pam) is predictable if ASLRis disabled. The adversary may also learn gVA_(pam) by using anyexisting attack method. In embodiments, the adversary may targetpoisoning TLB entries indexed by gVA_(pam). In one or more embodiments,the attacker process may first mmap a page with the virtual address tobe gVA_(pam). It is noted that gVA_(pam) is only used in P_(c), and theattacker process may assign this virtual address to a new instructionpage. The attacker process may then copy the same instruction page asthe victim into the new page and replace a few instructions ofpam_authenticate (e.g., offset 0x5b0-0x65f of the binary, starting withtest % rdi % rdi) with mov $0% eax and ret (0xb8 0x00 0x00 0x00 0x000xc3). The adversary may also schedule the attacker process on the samelogical core as the victim process, e.g., by changing the CPU affinityof the vCPU. The attacker process may then repeatedly access thisinstruction page in a loop to preserve the TLB entries.

(4) Bypass authentication. After the attacker process poisons the TLBentries of pam_authenticate( ) the adversary may directly resume P_(c)without a TLB flush. Recall that, in step (2), P_(c) was paused prior toa page table walk to resolve gPA_(pam). The adversary may resume P_(c)without handling this page table walk in order to force P_(c) to reusethe poisoned TLB entries. This way, when P_(c) calls pam_authenticate( )it will execute the instruction in the attacker's address space.Therefore, the function will directly return a 0 in EAX, thus, allowingan arbitrary user login.

2. Evaluation

Experimental settings comprise: an AMD EPYC™ 7251 CPU with 8 physicalcores; a SEV-ES VM downloaded from the AMD SEV repository; a host kernelversion sev-es-v3; a QEMU version sev-es-v12; an OVMF versionsev-es-v27; a victim VM comprising SEV-ES-enabled VMs with 4 vCPUs, 4 GBDRAM, and 30 GB disk storage; OpenSSH version OpenSSH_7.6 p1; andOpenSSL version 1.0.2n. The attack was repeated 20 times and all attackssuccessfully bypassed password authentication and could log in withincorrect passwords.

FIG. 5 is a flowchart illustrating a hypervisor-controlled attackprocess for manipulating a privileged victim process's execution tobypass password authentication, according to embodiments of the presentdisclosure. In one or more embodiments, the hypervisor-controlled attackprocess (500) for manipulating a privileged victim process's executionmay start by using an attacker process to perform steps comprising, inresponse to a victim process calling a target function whose virtualaddress has been mapped to a malicious code physical address, suspending(505) execution of the target function at the physical address. Thehypervisor may use one or more NPF events to learn the physical addressof the target function, e.g., by manipulating page table entries totrigger an NPF event.

In one or more embodiments, the attacker process is an unprivilegedprocess that may operate on a on a vCPU on a guest VM and obtain avirtual address of the victim process. Conversely, the victim process,e.g., a privileged child secure shell daemon process, may operate on aon a different vCPU. In one or more embodiments, the hypervisor mayschedule both vCPUs at a same logical core. In one or more embodiments,the attacker process may cause (510) the virtual address in one or moreTLB entries to point to the malicious code physical address to generateone or more poisoned TLB entries.

Once a hypervisor pauses the victim process, the attacker process maysuspend (515) a TLB flush at a subsequent VMEXIT event to access the oneor more poisoned TLB entries and execute the malicious code to bypass apassword authentication. In one or more embodiments, the attackerprocess may suspend the TLB flush, e.g., after a context switch betweenthe two vCPUs and before the victim process. Further, the hypervisor mayexecute CPUID instructions to trigger one or more VMEXIT events toinitiate transmission or receipt of covert data in a CPUID-based covertdata channel.

F. TLB Poisoning without Assisting Processes Embodiments

This section demonstrates that TLB poisoning attacks may work evenwithout employing an attacker process in the victim VM. Intuitively,when attacker and victim processes share similar virtual address spaces,TLB misuse may happen between these processes without direct control ofeither process.

One or more embodiments target fork( ), a system call that is used tocreate new processes and that is widely used in server-sideapplications, such as OpenSSH, sftp, Nginx, and Apache web server, toserve requests from different clients. Forked child processes have ahigh probability of sharing a very similar virtual memory area with amajority of their virtual address space layout overlapping. Even if theVM's administrator chooses to enable ASLR, the same VMA randomization isapplied to the parent process and all child processes, which gives theadversary the chance to conduct TLB poisoning despite the unpredictableVMA. This similarity of address spaces of forked processes has beenexploited in memory hijacking attacks.

Attack scenarios. Similar to the previous scenario that highlights a TLBpoisoning attack embodiment against an SSH server, Dropbear SSH, whichis a lightweight open-source SSH server written in C and releasedfrequently since 2003, was targeted. The more popular OpenSSH was notchosen because it alters its memory address space in all its childrenprocesses that serve incoming connections (by calling exec( )). However,this mechanism is typically only observed in OpenSSH and OpenBSD. Othernetwork applications like Dropbear SSH and Nginx do not change theirvirtual memory layout for different connections.

It was assumed that the targeted Dropbear SSH server application is freeof memory safety vulnerabilities and timing channel vulnerabilities. Itwas further assumed that the binary of the Dropbear Server applicationis known by the adversary, and that the username of a legitimate user isknown by the adversary—a practical assumption as usernames are notconsidered secrets. To simplify the attack, it was also assumed that thetwo processes are scheduled on two different vCPUs, which makes theattack easier to perform; otherwise, a VMCB-switching approachembodiment may be required.

1. Poison TLB Entries Between Connections Embodiments

Two SSH connections are considered: a connection from the adversary,which is served by the process P_(atk) that is forked from the DropSSHserver daemon, and another connection from a legitimate user, which isserved by the process P_(vic). One attack goal is to allow the attackerprocess to temporarily use the victim process's TLB entries andcircumvent password authentication.

Regular login procedures. After the login password packet is received bythe victim VM, P_(vic) calls svr_auth_password( ) to validate thepassword. As shown in Listing 2 below, the password encryption functionin the POSIX C library crypt( ) is called to generate a hash of theuser-provided password. The result may be stored in a buffer calledtestcrypt. The buffer storing the plaintext of the password may beimmediately freed. Then, the hash of the user-provided password may becompared with the stored value in the system file usingconstant_time_strcmp( ) which returns 0 if these two strings areidentical. If the user-provided password is correct, P_(vic) takes thecorrect-password branch, which calls send_msg_userauth_success( ).Otherwise, the incorrect password branch is taken.

Listing 2: Code snippet of svr_auth_password(). 1 void svr_auth_password(int valid_user ) { 2  char * passwdcrypt = NULL; 3  // store the cryptfrom / etc/ passwd 4  char * testcrypt = NULL; 5  // store the cryptgenerated from the password sent 6  ... 7  // ** Execution Point 1 (NPF)8  if (constant_time_strcmp (testcrypt, passwdcrypt) == 0)   { 9 //successful authentication 10  // ** Execution Point 2 (NPF) 11   send_msg_userauth_success (); 12   } ... 13 }

Attack overview. Test results demonstrate that by breaking the TLBisolation, the attacker process P_(atk) may bypass passwordauthentication by using even an incorrect password. Specifically, thevirtual addresses of the testcrypt buffer are usually the same for bothP_(atk) and P_(vic), which is empirically evaluated later. <gVA_(pwd),sPA_(pic)> denotes the TLB entry owned by P_(atk), which caches themapping from the virtual address of the testcrypt buffer to the systemphysical address that stores the hashed password used in P_(vic). Onegoal is to ensure the TLB entry <gVA_(pwd), sPA_(pic)> is not flushedwhen P_(atk) executes constant_time_strcmp( ). In this way, P_(atk) mayre-use the testcrypt of P_(vic) to circumvent password authentication.

Some Key Challenges. One key challenge in this attack is to ensure thatonly necessary TLB entries are preserved. Otherwise, later TLB entriesmay flush those necessary TLB entries. To address the challenge, the TLBpoisoning may be performed at the proper execution point. As shown inFIG. 3 , which depicts an overview of an exemplary attack steps tobypass password authentication, according to embodiments of the presentdisclosure, the adversary may locate the execution points right beforeand after the password authentication (e.g., constant_time_strcmp( ),which may be accomplished using the NPF controlled channels. It isassumed that the guest physical addresses of the instruction page wherethe svr_auth_password( ) and the constant_time_strcmp( ) are locatedare, respectively, gPA₁ and gPA₂. The adversary may first trap theattacker process in an empty loop when handling the NPF of gPA₂(execution point 1), which means P_(atk) is about to callconstant_time_strcmp( ). Then, the adversary does not interrupt untilP_(vic) also reaches the NPF of gPA₂ (execution point 1). When handlingthis NPF, the adversary may trigger a complete TLB flush. P_(vic) maythen continue execution until it finishes the password authenticationand try to return to svr_auth_password( ). An NPF of gPA₁ (executionpoint 2) is observed, and the adversary traps P_(vic). Meanwhile, theadversary may release the attacker process and skip the TLB flush. AllTLB entries used by P_(vic) during the execution ofconstant_time_strcmp( ) may thus be preserved in the TLB, including TLB(gVA_(pwd), sPA_(vic)). After the attacker process completesconstant_time_strcmp( ), passes the password check, and reachesexecution point 2, the adversary may trigger a complete TLB flush, e.g.,to avoid unnecessary TLB misuse, and release P_(vic). Both P_(atk) andP_(vic) may continue execution as normal, and no traces are left in thekernel message.

FIG. 6 is a flowchart illustrating a non-hypervisor-controlledtranslation lookaside buffer TLB poisoning process for attacking anetwork application, according to embodiments of the present disclosure.In one or more embodiments, process (600) for attacking a networkapplication may start by performing (605) an attacker process. Theattacker process may comprise clearing Present bits in an NPT, e.g., inresponse to a victim process being initialized, e.g., by an SSHconnection that represents a child process that has been forked from anSSH server daemon, for performing a login procedure. The attackerprocess may further comprise using an invalid password to initialize(610) a password authentication process, e.g., by using a differentsecond SSH connection.

In one or more embodiments, the victim process may operate on one vCPU,and the attacker process may operate on a different vCPU. Prior to theinvalid password being authenticated, the password authenticationprocess may be intercepted (615) and the victim process may resume,e.g., to populate (620) a TLB with TLB entries that point to a validpassword. In one or more embodiments, intercepting may comprise, inresponse to clearing the Present bits, triggering NPF events to locate aphysical address that comprises an instruction for a password comparisonfunction. Similarly, a password validation process may be intercepted byusing NPF events to locate a physical address that comprises aninstruction for the password validation function.

In one or more embodiments, in response to the victim processauthenticating the valid password, the victim process may be intercepted(625), and the attacker process may resume (630) without performing aTLB flush, thereby, preserving the TLB entries for use by the attackerprocess. The attacker process ma then reuse TLB entries, e.g., to readinstructions of the victim process or execute instructions of the victimprocess. In one or more embodiments, the TLB flush may involve a contextswitch between two virtual vCPUs or a world switch between a guestvirtual machine and a hypervisor.

In one or more embodiments, the TLB entries may be accessed (635) toascertain the valid password, e.g., by reading a hash of the validpassword. It is noted that once the valid password has been ascertainedand, e.g., prior to the victim process resuming execution, at least someof the TLB entries may be flushed in this manner.

2. End-to-end Attack Embodiments

An adversary may follow the following steps to carry out an end-to-endattack:

(1) Monitor network traffic. Even if the adversary cannot directly learnthe content of encrypted network packets, the adversary may inspectincoming and outgoing network packets through the unencrypted metadata(e.g., destination address, source address, or port number). Theadversary may continuously monitor network traffic to identify the SSHhandshake procedure. Once the adversary identifies a client_hello packetsent by a legitimate user, the adversary may trap that packet and send aclient_hello packet from a remote machine controlled by the adversary tothe victim VM. Once this client_hello packet reaches the victim VM, theadversary may resume processing the client_hello packet sent by thelegitimate user. Thus, the victim VM may receive two connectionrequests, one from the adversary and another from the legitimate user.

(2) Monitor fork( ) and gCR3 changes. Next, the adversary may locate thegCR3 of the forked child processes. During the victim VM's bootingperiod, the adversary may continuously monitor gCR3 changes, e.g., bysetting the CR3_WRITE_TRAP intercept bit in the VMCB. As a result, gCR3changes may cause an automatic VMEXIT with the new gCR3 value stored inVMEXIT EXITINFO. After receiving the two SSH connection packets, theDropbear Daemon may fork twice to generate child processes for theadversary's connection and the legitimate user's connection. The forkedchild process for the adversary's connection is referred to as P_(atk),whose gCR3 is gCR3_(atk). The forked child process for the legitimateuser's connection is referred to as P_(vic), whose gCR3 is gCR3_(vic).The adversary may identify gCR3_(atk) and gCR3_(vic) by correlating themwith the received client_hello packets.

(3) Monitor NPFs to locate the target gPAs. The adversary may try to login by sending an arbitrary password. The legitimate user logs in bysending a correct password. The adversary triggers NPFs by clearing thePresent bits in the NPT, when the encrypted SSH packets that contain thepasswords are observed. A sequence of NPFs for P_(atk) and a sequence ofNPFs for P_(vic) may be observed. The adversary may also collectadditional information (e.g., NPF EXITINFO2) along with the NPF VMEXITs,which reveals valuable information. For instance, the adversary maylearn that the NPF is caused by write/read access, user/kernel access,code read, or page table walks. The adversary may also clear (e.g.,periodically, every 50 NPFs) Present bits to finetune the NPF sequence.Since the Dropbear's binary is known to the adversary, the adversary maylearn the NPF patterns offline to locate the gPA of svr_auth_password( )(denoted gPA₁) and the gPA of the first instruction inconstant_time_strcmp( ) (denoted gPA₂). The features used in patternrecognition may be, for example, the sequence of NPFs and their errorcode. During an attack, the adversary may use the recognized pattern tolocate gPA₁ and gPA₂.

(4) Skip TLB flush. An adversary may continuously monitor P_(atk) andP_(vic). When observing the NPF of gPA₂ in P_(atk), the adversary maytrap P_(atk) in an empty loop and clear the Present bit of all pages.When observing the NPF of gPA₂ in P_(vic), the adversary may clear thePresent bit for all memory pages and perform a complete TLB flush. Theadversary may trap P_(vic) when observing the NPF of gPA₁. P_(atk) maythen resume, and the adversary may skip the TLB flush. P_(atk) uses thepreserved TLB entries from P_(vic) to read the password hash from thetestcrypto in the address space of P_(vic), which enables a successfullogin using an incorrect password. To avoid further TLB pollution, theadversary may then force a complete TLB flush and resume the victimprocess. Then, both P_(atk) and P_(vic) may resume regular execution.

3. Evaluation

Experiments were performed on a workstation with an AMD EPYC™ 7251Processor CPU with 8 SMT-enabled physical cores. The VMs, including thevictim VM and the training VMs used in this section were SEV-ES-enabledVMs with four vCPUs, 4 GB DRAM, and 30 GB disk storage. The OS software,QEMU (a machine emulator/hypervisor), and the UEFI (Unified ExtensibleFirmware Interface) image were the same as in Section E.2. ASLR wasenabled in the SEV-ES-enabled VMs by setting the parameter in/proc/sys/kernel/randomize_va_space to 2. The source code of Dropbearwas downloaded from Github (commit:846d38fe4319c517683ac3df1796b3bc0180be14). The Dropbear SSH Server wasconfigured as the default setting. The Dropbear SSH Server was bonded toPort 22. One minor non-default setting to assist the attack was thatP_(atk) and P_(vic) were forced to execute on different vCPUs of thevictim VM. It is noted that this setting improved the success rate ofthe attack but is not necessary in practical attacks.

Buffer address variation. The variation of the virtual address oftestcrypto under different connection ratios was first evaluated. In thetraining VM, the Dropbear server was modified to print the virtualaddress of testcrypto to the console after each connection. Then, asimple script was used to initiate new SSH connections, the correctpassword was sent to login, the virtual address of testcrypto wasobtained, and the current SSH connection was ended. In total, 120connections were collected. For the first 60 connections, the timeinterval between two contiguous connections was set to 0.1 second. Forthe second 60 connections, the time interval was set to 1 second. Asshown in FIG. 4A, when the time interval is set to 0.1 seconds, although3 different virtual addresses of the testcrypto are observed, thevirtual address of testcrypto remains the same in 57 of the total 60connections. As shown in FIG. 4B, when the time interval is set to 1second, the virtual address of testcrypto remains the same in 55 of thetotal 60 connections. The experimental results show that the virtualaddresses for testcrypto are relatively stable for differentconnections, which enables the adversary to poison the TLB entries ofthe testcrypto buffer between two connections.

Pattern matching. The pattern matching performance was evaluated asfollows: The above attack steps were repeated 100 times, and, each time,pattern matching was performed on-the-fly. In 98 of 100 trials, theadversary was able to correctly recognize the pattern and locate thegPA. The average time used to locate the pattern was 0.10137 secondswith a standard deviation of 0.02460 seconds.

End-to-end attacks. The success rate of end-to-end attacks was evaluatedas follows: The adversary conducted end-to-end attacks in the victim VMand used an incorrect password for SSH connections. The adversaryrepeated the attacks 20 times, and in 17 connections, the adversary wasable to log in with the incorrect password. Two reasons may account forthe 3 failed attempts. First, the reserved TLB entries may have beenevicted before use and, second, the presence of false positives inpattern matching. However, it is understood that an adversary may alwaysrepeat attacks the next time a legitimate user logs in.

G. Discussion and Countermeasure Embodiments

This section discusses, applications of TLB poisoning attacksembodiments on SEV-SNP, their differences compared to known attacks, andvarious countermeasures.

1. Comparison with Known Attacks

Previous works break confidentiality and/or integrity of SEV byreplacing unprotected I/O traffic, manipulating NPT mapping, and usingunauthenticated encryption. SEV-SNP may mitigate these by using aReversed Map table (RMP), which establishes a unique mapping between asystem physical address and either a guest physical address or ahypervisor physical address. The RMP also records the ownership of eachsystem physical address (e.g., a hypervisor page, a hardware page, or aSEV-SNP VM's page) as well as the ASID. For SEV-SNP VM, the RMP approachchecks the correctness and ownership after a nested page table walk.Once ownership is verified, mapping between the guest virtual addressand the system physical address may be cached in the TLB. This ownershipcheck prevents the hypervisor from remapping the guest physical addressto another system physical address and, thus, prevents attacks thatrequire manipulation of the NPT. Meanwhile, the RMP restricts thehypervisor's ability to write to the guest VM's memory page, whichmitigates attacks relying on unauthenticated encryption and unprotectedI/O operations.

In contrast, this patent document demystifies how TLB isolation isperformed in SEV and demonstrates security risks thathypervisor-controlled TLB flushes may pose. TLB poisoning attacks bythemselves do not rely on the known vulnerabilities of SEV and SEV-ES,such as lack of authenticated memory encryption, lack of NPT protection,and lack of I/O protection, nor does RMP alone prevent TLB poisoningattacks.

2. Countermeasure Embodiments

TLB poisoning attacks affect SEV and SEV-ES servers, including all firstand second generation EPYC™ server CPUs (e.g., Zen 1 and Zen 2architectures). Older processors may use a microcode patch to enforce aTLB flush during VMRUN for all SEV/SEV-ES vCPUs. From the software side,to mitigate TLB poisoning attacks, it may be recommended that allnetwork-related applications (e.g., HTTPS, FTP, and SSH server) useexec( ) to ensure a completely new address space for a new connection.

re-exec( ) Scheme Embodiments. In network applications, the sshd processuses the fork( ) commend to generate a child process to handle a newconnection. The created child processes produced by fork mechanismtypically have VMAs that are similar to those of their parent processes.An adversary may exploit this circumstance to facilitate TLB poisoningattacks.

In one or more countermeasure embodiments, TLB poisoning attacks may bemitigated by taking advantage of a re-exec scheme to introduce freshnessinto the VMA of a child processes to cause VMAs of child processes to bedifferent from those of parent processes. In one or more embodiments,this may be accomplished by the following process that utilizes are-exec( ) scheme to successfully thwart TLB poisoning attacks.

Step 1—A network application's daemon process (denoted P_(d)), which maybe launched during system boot, may run in the background and listen toconnections on certain ports, e.g., port 22 for SSH. The address spaceof daemon process P_(d) may be defined in the kernel by VMA datastructures. In one or more embodiments, upon receiving a connectionrequest, the daemon process P_(d) may fork an SSHD child process, P_(c).

Step 2—P_(c) itself may call an exec( ) command, i.e., a re-exec( )After re-exec( ) once ASLR is enabled, e.g., on a Linux system, ASLR mayrandomize the VMA of the child process, P_(c), to ensure a new addressspace for the connection. Thus, as illustrated in Section F.1 above,virtual addresses of the secret “testcrypt” may also vary for differentconnections, thereby, successfully thwarting a TLB poisoning attack.

Step 3—The daemon process P_(d) may repeat Step 2, e.g., when there is anew connection.

It is noted that the above process is only one example of how toimplement the re-exec scheme according to various embodiments herein. Asa person of skill in the art will appreciate that other implementationare possible without deviating from the true spirit of this particularcountermeasure.

FIG. 7 is countermeasure against TLB poisoning attacks, according toembodiments of the present disclosure. In one or more embodiments, inresponse to a connection request in a network-related application, achild process that has been forked from a daemon process may be received(705), the daemon process occupying a first address space in a firstVMA. A command of the child process may be executed (710), such that thecommand causes the child process to occupy a second address space in asecond VMA that is different from the first address space.Advantageously, this prevents a TLB poisoning attack process fromaccessing and manipulating a TLB entry associated with the secondaddress space.

It is understood that the command may be an exec( ) command, and thedaemon process that forks the child process, e.g., an SSHD childprocess, may also call an exec( ) command. Further, the daemon process,which may be launched during system boot, may run in the background andmonitor connections on one or more ports.

In one or more embodiments, the virtual address associated with a systemphysical address of a buffer that comprises a hash of a user-providedpassword may be altered to cause the virtual address to be different fordifferent connections. Finally, the child process and/or the daemonprocess may have an address space that is defined in a kernel, e.g., aLinux kernel that may enable ASLR to perform the randomization.

TLBreach Flushing Embodiments. In one or more embodiments, TLB poisoningattacks may be mitigated by utilizing a hardware-enforced TLB flushscheme. For example, each time a VMRUN occurs, hardware may enforce aTLB flush for SEV-enabled VM's ASID. In this manner, previous TLBentries from other processes inside the VM may be flushed to prevent apotential TLB misuse. In embodiments discussed above in Section E, thehardware may force a TLB flush even if an adversary chooses to skip theTLB in Step (4) in Section E.1b) to ensure that those poisoned TLBentries are forced to be evicted before the victim process continuesexecution. Similarly, in embodiments discussed above in Section F, tothwart a TLB poisoning attack, the hardware may force a TLB flush evenif an adversary chooses to skip the TLB in Step (4) in Section F.2.

FIG. 8 is a countermeasure against a TLB poisoning attack utilizing ahardware-enforced TLB flush process for mitigating hypervisor-controlledTLB poisoning attacks, according to embodiments of the presentdisclosure. In one or more embodiments, in response to an event (e.g.,VMRUN) in an SEV-enabled victim VM, process (800) may cause hardware toenforce (805) a TLB flush for the victim VM's ASID. Advantageously, thisprevents a victim process, especially a privileged victim process, fromaccessing one or more poisoned TLB entries that may, otherwise, bemanipulated by an unprivileged attacker process to bypass a passwordauthentication. As a result, the victim process may be prevented fromusing the poisoned TLB entries and executing malicious code, e.g., bypreventing a hypervisor from using the attacker process to cause thevictim process to access and use poisoned TLB entries to read or executeof the victim process.

In one or more embodiments, the hypervisor may be prevented fromcontrolling the attacker process to cause the victim process to usepoisoned TLB entries after, e.g., a context switch between the firstvCPU and the second vCPU or a VM exit event. It is understood that amalicious hypervisor may launch an application that comprises one ormore security vulnerabilities that enable remote access to the attackerprocess, e.g., an attacker process that operates on a vCPU on the victimVM, while the victim process may operate on a different vCPU on thevictim VM. It is further understood that, in one or more embodiments,the unprivileged attacker process may operate in the victim VM and beassociated with an unprivileged user account on the victim VM.

In one or more embodiments, an attacker process may generate poisonedTLB entries by performing steps comprising (1) mapping a virtual addressof a target function to a malicious code physical address; (2) inresponse to the victim process calling the target function, halting thevictim process to access a virtual address in one or more TLB entries;and (3) manipulating the virtual address to point to the malicious codephysical address.

H. Some Related Work

Several design flaws of AMD SEV have been reported since its debut in2016, including unencrypted VMCB, unprotected I/O interface, unprotectedmemory mapping, unauthenticated memory encryption, and unauthenticatedASID.

Unencrypted VMCB. An unencrypted VMCB leaves SEV vulnerable to numerousattacks since the VM's states (e.g., registers) are saved in plaintextduring a traditional world switch in AMD-V under the assumption that thehypervisor is trusted. To mitigate these attacks, AMD released SEV-ES,the second generation of SEV that encrypts the sensitive portions of theVMCB and stores them into the VMSA during the world switch.

Lack of memory integrity. Most of other attacks can work on SEV-ES. Someattacks involve unencrypted I/O operations on SEV and SEV-ES. On SEV,peripheral devices (e.g., disk, the network interface card) are notsupported to directly read/write guest VMs' memory with thecorresponding K_(vek). Thus, an additional buffer area is reserved andmaintained by the guest VM, which provides an interface for thehypervisor to generate encryption/decryption oracles during I/Otransmission. Other attacks involve memory mapping problems caused byhypervisor-controlled nested page tables on SEV.

Unauthenticated ASID. Crossline attacks involve ASID misuse and the“Security-by-Crash” principle of AMD SEV and SEV-ES. ASID is used astags in TLB entries and cache lines, and also the identifier of memoryencryption keys in AMD-SP. However, the hypervisor is in charge of theASID management. AMD relies on the Security-by-Crash principle toprevent ASID misuse; it is expected that an incorrect ASID willimmediately crash the VM. However, it has been shown showed that byassigning the ASID of a victim VM to a helper VM, an adversary mayextract the victim VM's arbitrary memory block with the PTE format.Crossline attacks are stealthy, but NPT page remapping is stillrequired.

Page-fault side channels. Page-fault side channels are widely used inmany attacks against SEV. The guest VM maintains its own guest pagetable, which transfers guest virtual address to guest physical addressand is encrypted and protected by SEV. The lower nested page table istransparent to and maintained by an untrusted hypervisor. The hypervisorcan easily track the victim VM's execution paths by clearing the Presentbit in the lower NPT pages. Moreover, NPFs also reveal valuableinformation to the hypervisor (e.g., write/read access anduser/privileged access). Such information may be actively gathered bythe hypervisor and used to locate both the time point and the physicaladdress of sensitive data. The controlled-channel methods in theoryshould still work on SEV-SNP.

I. Computing System Embodiments

In one or more embodiments, aspects of the present patent document maybe directed to, may include, or may be implemented on one or moreinformation handling systems (or computing systems). An informationhandling system/computing system may include any instrumentality oraggregate of instrumentalities operable to compute, calculate,determine, classify, process, transmit, receive, retrieve, originate,route, switch, store, display, communicate, manifest, detect, record,reproduce, handle, or utilize any form of information, intelligence, ordata. For example, a computing system may be or may include a personalcomputer (e.g., laptop), tablet computer, mobile device (e.g., personaldigital assistant (PDA), smart phone, phablet, tablet, etc.), smartwatch, server (e.g., blade server or rack server), a network storagedevice, camera, or any other suitable device and may vary in size,shape, performance, functionality, and price. The computing system mayinclude random access memory (RAM), one or more processing resourcessuch as a CPU or hardware or software control logic, read only memory(ROM), and/or other types of memory. Additional components of thecomputing system may include one or more drives (e.g., hard disk drive,solid state drive, or both), one or more network ports for communicatingwith external devices as well as various input and output (I/O) devices,such as a keyboard, mouse, touchscreen, stylus, microphone, camera,trackpad, display, etc. The computing system may also include one ormore buses operable to transmit communications between the varioushardware components.

FIG. 9 depicts a simplified block diagram of an information handlingsystem (or computing system), according to embodiments of the presentdisclosure. It will be understood that the functionalities shown forsystem 900 may operate to support various embodiments of a computingsystem—although it shall be understood that a computing system may bedifferently configured and include different components, includinghaving fewer or more components as depicted in FIG. 9 .

As illustrated in FIG. 9 , the computing system 900 includes one or moreCPUs 901 that provides computing resources and controls the computer.CPU 901 may be implemented with a microprocessor or the like, and mayalso include one or more graphics processing units (GPU) 902 and/or afloating-point coprocessor for mathematical computations. In one or moreembodiments, one or more GPUs 902 may be incorporated within the displaycontroller 909, such as part of a graphics card or cards. Thy system 900may also include a system memory 919, which may comprise RAM, ROM, orboth.

A number of controllers and peripheral devices may also be provided, asshown in FIG. 9 . An input controller 903 represents an interface tovarious input device(s) 904. The computing system 900 may also include astorage controller 907 for interfacing with one or more storage devices908 each of which includes a storage medium such as magnetic tape ordisk, or an optical medium that might be used to record programs ofinstructions for operating systems, utilities, and applications, whichmay include embodiments of programs that implement various aspects ofthe present disclosure. Storage device(s) 908 may also be used to storeprocessed data or data to be processed in accordance with thedisclosure. The system 900 may also include a display controller 909 forproviding an interface to a display device 911, which may be a cathoderay tube (CRT) display, a thin film transistor (TFT) display, organiclight-emitting diode, electroluminescent panel, plasma panel, or anyother type of display. The computing system 900 may also include one ormore peripheral controllers or interfaces 905 for one or moreperipherals 906. Examples of peripherals may include one or moreprinters, scanners, input devices, output devices, sensors, and thelike. A communications controller 914 may interface with one or morecommunication devices 915, which enables the system 900 to connect toremote devices through any of a variety of networks including theInternet, a cloud resource (e.g., an Ethernet cloud, a Fiber Channelover Ethernet (FCoE)/Data Center Bridging (DCB) cloud, etc.), a localarea network (LAN), a wide area network (WAN), a storage area network(SAN) or through any suitable electromagnetic carrier signals includinginfrared signals. As shown in the depicted embodiment, the computingsystem 900 comprises one or more fans or fan trays 918 and a coolingsubsystem controller or controllers 917 that monitors thermaltemperature(s) of the system 900 (or components thereof) and operatesthe fans/fan trays 918 to help regulate the temperature.

In the illustrated system, all major system components may connect to abus 916, which may represent more than one physical bus. However,various system components may or may not be in physical proximity to oneanother. For example, input data and/or output data may be remotelytransmitted from one physical location to another. In addition, programsthat implement various aspects of the disclosure may be accessed from aremote location (e.g., a server) over a network. Such data and/orprograms may be conveyed through any of a variety of machine-readablemedium including, for example: magnetic media such as hard disks, floppydisks, and magnetic tape; optical media such as compact discs (CDs) andholographic devices; magneto-optical media; and hardware devices thatare specially configured to store or to store and execute program code,such as application specific integrated circuits (ASICs), programmablelogic devices (PLDs), flash memory devices, other non-volatile memory(NVM) devices (such as 3D XPoint-based devices), and ROM and RAMdevices.

Aspects of the present disclosure may be encoded upon one or morenon-transitory computer-readable media with instructions for one or moreprocessors or processing units to cause steps to be performed. It shallbe noted that the one or more non-transitory computer-readable mediashall include volatile and/or non-volatile memory. It shall be notedthat alternative implementations are possible, including a hardwareimplementation or a software/hardware implementation.Hardware-implemented functions may be realized using ASIC(s),programmable arrays, digital signal processing circuitry, or the like.Accordingly, the “means” terms in any claims are intended to cover bothsoftware and hardware implementations. Similarly, the term“computer-readable medium or media” as used herein includes softwareand/or hardware having a program of instructions embodied thereon, or acombination thereof. With these implementation alternatives in mind, itis to be understood that the figures and accompanying descriptionprovide the functional information one skilled in the art would requireto write program code (i.e., software) and/or to fabricate circuits(i.e., hardware) to perform the processing required.

It shall be noted that embodiments of the present disclosure may furtherrelate to computer products with a non-transitory, tangiblecomputer-readable medium that have computer code thereon for performingvarious computer-implemented operations. The media and computer code maybe those specially designed and constructed for the purposes of thepresent disclosure, or they may be of the kind known or available tothose having skill in the relevant arts. Examples of tangiblecomputer-readable media include, for example: magnetic media such ashard disks, floppy disks, and magnetic tape; optical media such as CDsand holographic devices; magneto-optical media; and hardware devicesthat are specially configured to store or to store and execute programcode, such as ASICs, PLDs, flash memory devices, other non-volatilememory devices (such as 3D XPoint-based devices), and ROM and RAMdevices. Examples of computer code include machine code, such asproduced by a compiler, and files containing higher level code that areexecuted by a computer using an interpreter. Embodiments of the presentdisclosure may be implemented in whole or in part as machine-executableinstructions that may be in program modules that are executed by aprocessing device. Examples of program modules include libraries,programs, routines, objects, components, and data structures. Indistributed computing environments, program modules may be physicallylocated in settings that are local, remote, or both.

One skilled in the art will recognize no computing system or programminglanguage is critical to the practice of the present disclosure. Oneskilled in the art will also recognize that a number of the elementsdescribed above may be physically and/or functionally separated intomodules and/or sub-modules or combined together.

It will be appreciated to those skilled in the art that the precedingexamples and embodiments are exemplary and not limiting to the scope ofthe present disclosure. It is intended that all permutations,enhancements, equivalents, combinations, and improvements thereto thatare apparent to those skilled in the art upon a reading of thespecification and a study of the drawings are included within the truespirit and scope of the present disclosure. It shall also be noted thatelements of any claims may be arranged differently including havingmultiple dependencies, configurations, and combinations.

What is claimed is:
 1. A hardware-enforced TLB flush method forthwarting hypervisor-controlled TLB Poisoning attacks, the methodcomprising: in response to a VMRUN event in a victim virtual machine(VM) that is Secure Encrypted Virtualization (SEV)-enabled, usinghardware to cause a translation lookaside buffer (TLB) flush for thevictim VM's address space identifier (ASID) to prevent a victim processfrom accessing one or more poisoned TLB entries, which have beenmanipulated by an attacker process to bypass a password authentication,the TLB flush preventing the victim process from using the one or morepoisoned TLB entries to execute malicious code.
 2. The method accordingto claim 1, wherein the victim process is a privileged victim process,and the attacker process is an unprivileged attacker process, whichoperates in the victim VM and is associated with an unprivileged useraccount on the victim VM.
 3. The method according to claim 1, whereinpreventing the attacker process from executing the malicious codecomprises preventing a hypervisor from using the attacker process tocause the victim process to access and use the one or more poisoned TLBentries to execute instructions.
 4. The method according to claim 3,wherein the hypervisor launches an application that comprises one ormore security vulnerabilities that enable remote access to the attackerprocess.
 5. The method according to claim 1, wherein the attackerprocess operates on a first virtual CPU (vCPU) on the victim VM, and thevictim process operates on a second vCPU on the victim VM.
 6. The methodaccording to claim 5, wherein the TLB flush prevents a hypervisor fromcontrolling the attacker process to cause the victim process to use theone or more poisoned TLB entries after at least one of a context switchbetween the first vCPU and the second vCPU or a VM exit event.
 7. Themethod according to claim 1, wherein the attacker process generates theone or more poisoned TLB entries by performing steps comprising: mappinga virtual address of a target function to a malicious code physicaladdress; in response to the victim process calling the target function,halting the victim process to access the virtual address in one or moreTLB entries; and manipulating the virtual address to point to themalicious code physical address.
 8. The method according to claim 1,wherein the TLB flush prevents a hypervisor from using the poisoned TLBentries to read instructions of the victim process.
 9. A non-transitorycomputer-readable medium or media comprising one or more sequences ofinstructions which, when executed by at least one processor, causessteps for thwarting translation lookaside buffer (TLB) poisoningattacks, the steps comprising: in response to a VMRUN event in a victimvirtual machine (VM) that is Secure Encrypted Virtualization(SEV)-enabled, using hardware to cause a translation lookaside buffer(TLB) flush for the victim VM's address space identifier (ASID) toprevent a victim process from accessing one or more poisoned TLBentries, which have been manipulated by an attacker process to bypass apassword authentication, the TLB flush preventing the victim processfrom using the one or more poisoned TLB entries to execute maliciouscode.
 10. The non-transitory computer-readable medium or media accordingto claim 9, wherein the victim process is a privileged victim process,and the attacker process is an unprivileged attacker process, whichoperates in the victim VM and is associated with an unprivileged useraccount on the victim VM.
 11. The non-transitory computer-readablemedium or media according to claim 9, wherein preventing the attackerprocess from executing the malicious code comprises preventing ahypervisor from using the attacker process to cause the victim processto access and use the one or more poisoned TLB entries to executeinstructions.
 12. The non-transitory computer-readable medium or mediaaccording to claim 9, wherein the attacker process operates on a firstvirtual CPU (vCPU) on the victim VM, and the victim process operates ona second vCPU on the victim VM.
 13. The non-transitory computer-readablemedium or media according to claim 12, wherein the TLB flush prevents ahypervisor from controlling the attacker process to cause the victimprocess to use the one or more poisoned TLB entries after at least oneof a context switch between the first vCPU and the second vCPU or a VMexit event.
 14. The non-transitory computer-readable medium or mediaaccording to claim 9, wherein the TLB flush prevents a hypervisor fromusing the poisoned TLB entries to read instructions of the victimprocess.
 15. A system for thwarting translation lookaside buffer (TLB)poisoning attacks comprising: one or more processors; and anon-transitory computer-readable medium or media comprising one or moresets of instructions which, when executed by at least one of the one ormore processors, causes steps to be performed comprising: in response toa VMRUN event in a victim virtual machine (VM) that is Secure EncryptedVirtualization (SEV)-enabled, using hardware to cause a translationlookaside buffer (TLB) flush for the victim VM's address spaceidentifier (ASID) to prevent a victim process from accessing one or morepoisoned TLB entries, which have been manipulated by an attacker processto bypass a password authentication, the TLB flush preventing the victimprocess from using the one or more poisoned TLB entries to executemalicious code.
 16. The system according to claim 15, wherein the victimprocess is a privileged victim process, and the attacker process is anunprivileged attacker process, which operates in the victim VM and isassociated with an unprivileged user account on the victim VM.
 17. Thesystem according to claim 15, wherein preventing the attacker processfrom executing the malicious code comprises preventing a hypervisor fromusing the attacker process to cause the victim process to access and usethe one or more poisoned TLB entries to execute instructions.
 18. Thesystem according to claim 17, wherein the hypervisor launches anapplication that comprises one or more security vulnerabilities thatenable remote access to the attacker process.
 19. The system accordingto claim 15, wherein the attacker process operates on a first virtualCPU (vCPU) on the victim VM, and the victim process operates on a secondvCPU on the victim VM.
 20. The system according to claim 19, wherein theTLB flush prevents a hypervisor from controlling the attacker process tocause the victim process to use the one or more poisoned TLB entriesafter at least one of a context switch between the first vCPU and thesecond vCPU or a VM exit event.